DM-46158: Add workflow to perform schema comparisons and check for changes to deployed schemas #277
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
JeremyMcCormick
changed the title
JeremyMcCormick
force-pushed
the
tickets/DM-46158
branch
6 times, most recently
from
eb88b7e to
80b68f4
Compare
JeremyMcCormick
force-pushed
the
tickets/DM-46158
branch
3 times, most recently
from
fcba30b to
72a9e39
Compare
JeremyMcCormick
force-pushed
the
tickets/DM-46158
branch
3 times, most recently
from
0d2da16 to
56b078f
Compare
JeremyMcCormick
force-pushed
the
tickets/DM-46158
branch
9 times, most recently
from
d0b379f to
cb91907
Compare
JeremyMcCormick
changed the title
gpdf
requested changes
Dec 6, 2024
There was a problem hiding this comment.
Two significant changes:
- Trigger only on PRs, and do the diff against the base ref of the PR
- Modify the .yaml changes on the PR to have one file with a non-schema-changing change to a deployed schema, one with a schema-changing change to a deployed schema, and one with a schema-changing change to a non-deployed schema
Following the next review of this PR:
- all three .yaml file changes must be removed from the PR before merging.
JeremyMcCormick
force-pushed
the
tickets/DM-46158
branch
3 times, most recently
from
8e21f7c to
319d41a
Compare
JeremyMcCormick
force-pushed
the
tickets/DM-46158
branch
from
319d41a to
8b33d32
Compare
gpdf
approved these changes
Dec 10, 2024
JeremyMcCormick
force-pushed
the
tickets/DM-46158
branch
3 times, most recently
from
a99d2d5 to
bede84e
Compare
|
I've corrected a problem where there was an error if no changes had occurred. And I have updated the workflow so that it skips subsequent skips when there are no changes. I tested these updates for both cases by also re-committing and testing on the dummy schema changes, which I then dropped, again. |
JeremyMcCormick
force-pushed
the
tickets/DM-46158
branch
3 times, most recently
from
492fe22 to
56930a6
Compare
JeremyMcCormick
force-pushed
the
tickets/DM-46158
branch
from
56930a6 to
6a9c461
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a workflow that will check for changes in schemas using the
felis diffcommand. One comparison is performed using thedeepdiffcomparator, which will show all changes, including ones to metadata fields, e.g.,ivoa_ucd. Another comparison step will usealembic, which will only show updates that affect the relational data model.Finally, a comparison using the
alembicmode will be performed on each schema from a list of "deployed" schema YAML files (which is also added by this PR). An error will be raised if any of them have changed (dp02_dc2, etc.).A preliminary step in the workflow uses
git diffto detect which files were changed. Files which were not changed in the repository need not be checked.Blocked by lsst/felis#109This is merged now. ^^^^